System and Method For Search and Display of Content in the Form of Audio, Video or Audio-Video

ABSTRACT

The invention relates to the field of computer search on a network and is particularly directed to searching for and displaying content in the form of audio, video and/or audio-video (“A-V”) results wherein the content that results from the search includes a textual transcript associated with the content. A system and method provide seamless delivery of audio, video and A-V results that are auto-populated into a media player on a computer system.

RELATED UNITED STATES APPLICATION DATA

This application is a continuation-in-part of U.S. patent applicationSer. No. 12/876,026 entitled “A System and Method for Search and Displayof Audio-Video Content,” filed on Sep. 3, 2010, which claims prioritybenefit from U.S. Provisional Application No. 61/240,081, filed on Sep.4, 2009.

BACKGROUND

It is an everyday occurrence for most of us to use an internet searchtool on a computer or other electronic device connected to a network tofind any conceivable information that is of interest to us. As more andmore information is posted on the internet, as well as on privatenetworks, the need to efficiently search and access that information hasgrown exponentially. Search has become a huge business dominated byGoogle, Inc. of Mountain View, Calif. In response to a search string orquery input by a user, a search engine such as Google's considers a hostof factors before it delivers a prioritized list of results. Othercompanies offering similar search services to users hungry forinformation include Microsoft, Yahoo, and IAC among others.

The most successful search engines use keywords typed in by the user tocomb through millions of web pages in search of relevant informationthat the search engine algorithms are programmed to return. Fortext-based web pages, this type of searching has proven enormouslysuccessful. As bandwidth for internet users has expanded, web pages havebecome far more sophisticated and dynamic, now hosting many differentformats such as audio, video and/or A-V recordings that can be played bya user through their internet connection. For purposes of thisspecification, the terms audio, video, A-V, media and multimedia are allfiles containing content of the different types that may be streamedlive or played back from a recording. It should be further understoodthat any of these different types of content lend themselves to storageand play back in different file formats which will be discussedthroughout the specification. It should be understood that the use of“audio,” “video,” “A-V,” “media” or “multimedia” individually ortogether throughout this specification is intended to cover any one ormore of these content types in one or more formats where appropriate.

As these different types of media have proliferated across the web andbecome standard on internet web pages, the search engines have failed tokeep up in terms of the ability to search the content of non-text-basedformats such as audio, video and A-V recordings. While text-based pagesare predominantly in one of a few formats such as XML, HTML, DOC, or PDFthat allow strings of characters to be identified and compared,searching the content of an audio, video or A-V recording is far morechallenging.

Of the media players available on the market today, none allow for asimple, seamless searching and synchronized playing of a selectedsegment of audio, video and/or A-V content directly from the popularsearch engines.

However, once the audio track of recorded material that contains speech,sounds or visual cues is converted to text, that text is searchable bysearch engines available in the market today. It is worth noting thatthe actual search results produced by a search engine analyzing atranscribed audio, video or A-V track is only as accurate as thespeech-to-text, sound-to-text or visual-to-text transcription that isperformed.

While text based search engines are widely available for network use,audio, video and/or A-V search tools are not. An example of a limitedcapability search tool for video is the experimental video search“gadget” (formerly the “Gaudi gadget”) provided by Google of MountainView, Calif. On Google's web pages dedicated to political videos, a usermay search the videos on a limited set of web pages by entering a searchterm in the search query box. The results from the limited set of webpages are listed and can be selected by a user to be taken to thebeginning of the video that contains that term.

There are various companies offering software that uses algorithms toautomatically produce transcripts from audio, which are thensynchronized with video containing the audio. One such product isMetaPlayer produced by RAMP, Inc. of Woburn, Mass. (formerly Everyzing,Inc. of Cambridge, Mass.) Companies like RAMP offer search within anindividual video by searching for a text string matching text in thetranscription. When a search string is entered for a particular video,results are listed. When a result is selected, the video is played fromthat occurrence of the searched string.

The present invention recognizes the desirability of producing advancedsearch capabilities for audio-only and/or audio-video content, as wellas the use of those same capabilities enhanced with accuratetranscription and synchronization.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 a is an internet screen shot of a search page;

FIG. 1 b is an internet screen shot of a search page including a searchstring input by a user into a query box;

FIG. 1 c is an internet screen shot of a search result page returned bya search engine in response to the search string entered in FIG. 1 b;

FIG. 1 d is an internet screen shot of a web page to which a user istaken after clicking on a highlighted search result in FIG. 1 c;

FIG. 2 a is an internet screen shot of a YouTube web page;

FIG. 2 b is an internet screen shot of the YouTube web page including asearch string input by a user;

FIG. 2 c is an internet screen shot of a search result page returned bythe YouTube search engine in response to the search string entered inFIG. 2 b;

FIG. 2 d is an internet screen shot of a web page to which a user istaken after clicking on the highlighted search result in FIG. 2 c;

FIG. 3 a is an internet screen shot of a search engine web pageincluding a search string input by a user intended to narrow resultsreturned by putting quotation marks around the search string, inaccordance with the present invention;

FIG. 3 b is an internet screen shot of a search result page returned bya search engine in response to the search string entered in FIG. 3 a, inaccordance with the present invention;

FIG. 3 c is an internet screen shot of a media player where the selectedsearch result highlighted in FIG. 3 a has been selected andautomatically passed through to the search box in the media player andwhere audio-video content is queued to an occurrence of the searchstring, in accordance with the present invention;

FIG. 3 d is an internet screen shot of a help screen for a sample mediaplayer of the present invention, which explains the features and how tooperate the player;

FIG. 4 is a flow chart of a sample embodiment of a media player processin accordance with the present invention;

FIG. 5 is a flow chart of a sample embodiment of media player softwarecode in accordance with the present invention; and

FIG. 6 is a block diagram of a computer system and network of thepresent invention connected to permit a user to perform searches andreview media results of audio, video and/or A-V content.

DETAILED DESCRIPTION

Searching the internet or any other network for information using acomputer or another electronic device is performed using any of a numberof available search tools. As any person familiar with the internetknows, it is a simple exercise to enter a query either by typing it inon a keyboard, entering it by a voice command, or otherwise inputting itto the system through a user interface, and within a second or less havea list of results, descriptions or “hits” with links to web pages thatare related to the key words typed into the user query. The user canclick any hit to drill down into the search results and determinewhether any particular hit is of interest. Searching in this manner canassist a user in an infinite number of ways in finding informationrelated to products to purchase, product prices, news clips, sportsscores, biographical information, historical data, stock prices or justabout anything else that an individual can imagine or think of.

It should be understood that throughout this description, the term“internet” refers to the world wide web which is a ubiquitous networkaccessed by users all over the world. It should be understood that othernetworks whether public or private may utilize the invention. Therefore,any use of the term “internet” should be considered synonymous with theterm “network.” Further, throughout this description, the term“computer” refers to an electronic device capable of connecting to anetwork and which now has, or will in the future have capabilities forconducting search on the network to which it is connected, among otherfunctionalities. For purposes of this description, the term “computer”should be considered synonymous with any electronic or computing devicewith capabilities for conducting search on a network, including but notlimited to a(n): (a) desktop computer, (b) laptop computer, (c) server,(d) tablet or netbook computer, (e) mobile device such as a smartphone,(f) a wearable device such as a smartwatch, (g) computing deviceintegrated into another device such as eyeglasses, an automobile, (h) asmart TV, (i) a game console such as an Xbox®, a Playstation® or a Wii®,(j) electronic music player, electronic reader, camera or otherdedicated device with network connectivity, or (k) a portable gameplayer such as a PSP® or a GameBoy®. On any of these devices, it shouldbe understood that a search query may be entered using a keyed input, avoice command, or any other method of data entry. For purposes of thisdescription, use of the term “type” shall be synonymous with “enter” andshall include any data entry whether by typing, voice command or anyother method for inputting a search query. In addition, a “computer” or“computing device” shall be understood to be connectable to a network,and actions performed on the computer or computing device may includelocal actions conducted on the device itself or activities conductedacross the network to which it is connected, including but not limitedto environments such as server-based computing, cloud computing,cloud-based computing, virtualization and the use of virtual servers,any of which may be running applications interfacing with the computeror computing device, before, during and after search activities. It willbe understood that the computing device may operate in any of theseenvironments using the invention.

An example of a search page is shown in FIG. 1 a. FIG. 1 a is a screenshot 101 of a search page provided by Google, Inc. of Mountain View,Calif. Search page 101 contains a number of other elements such as links105 in the upper left corner for taking the user to other applicationsoffered by Google. These applications are Images 105 a, Video 105 b,Maps 105 c, News 105 d, Shopping 105 e, Gmail 105 f and more 105 g thatwill list other links if selected. A user may enter their query into aquery box 107 and then click on Google Search box 109, I'm Feeling Luckybox 111 or simply press the enter key on the keyboard (see FIG. 6) tostart the search. The screenshot of FIG. 1 a is shown using the Firefoxbrowser offered by Mozilla of Mountain View, Calif. It should beunderstood that a user can use another browser or search engine such asInternet Explorer or Bing offered by Microsoft Corporation of Redmond,Wash. or any other browser enabling a user to access and search webpages on the internet.

FIG. 1 b shows the search page of FIG. 1 a where the query or searchstring “An epiphany hit me. You gotta make something good” has beentyped into the query box 107. Once the search string is entered, thesearch tool performs a search across the internet employing algorithmsprogrammed to return search results that are deemed most relevant to theentered string. Some search tools are better at delivering relevantresults than others, and different search tools may deliver differentresults on the same string. Users may find a favorite search tool andstick with it exclusively or may employ multiple search tools in theirquest to find the most pertinent information.

Search queries of different types are well known, including searcheswith different types of filtering for targeting improved search results.A search may include one or more words, or it may be structured with agrouping of words in a particular order. For example, many searchengines recognize a set of words or other textual symbols surrounded byquotes to mean that the user is searching for the exact search string asinput. In some cases, the search engine includes algorithms thatconsider the query and may suggest other options, such as, for example,where the user has input a word in the string that is misspelled orotherwise not recognized. Other search engines have specific queryformats that the user must follow. These may include using instructionswithin the search string or identifying a particular type of database tobe searched. An example of an instruction set for performing searchesusing Google's search engine may be found at:

https://support.google.com/websearch/answer/142143?hl=en

Further, search engines allow filtering or refinement either before orafter a set of results have been returned to the user. For example, auser may require that the search be limited to images or maps, or thatthe results be limited to retail establishments offering goods orservices for sale. Using these different search tools, a user may limitthe results delivered to achieve a better search experience. It shouldbe understood that the present invention may incorporate any and all ofthese search techniques.

FIG. 1 c shows the results 113 a-c of a traditional internet searchdelivered by the Google search tool for the string “An epiphany hit me.You gotta make something good” entered into query box 107. As mentionedabove, the user can click on any of the returned results or “hits.” Eachhit is hyperlinked to the web page containing the searched-forinformation and allows the user to be taken to that page by simplyclicking on the hit. Additional hits are available to the user byclicking on the “O”s 115 or numbers under the O's 119 at the bottom ofFIG. 1 c.

FIG. 1 d shows a resulting web page from the website TED.com, after theuser clicks on the highlighted result 113 a in the listed searchresults. The information displayed on the TED web page 121 is presumablysomething of interest to the user based on the search string theyentered into the query box 107. If not, the user may simply hit the“back” button 123 on the browser window and go back to search results113. Once back at the search result page FIG. 1 c, the user may select adifferent result from the list of search results 113 and pull up thecorresponding web page to determine whether that web page containsinformation of interest.

This process may be repeated numerous times until the user gathers theinformation that they are looking for. To date, this process has becomethe standard for searching the internet for information. One area ofinterest that is largely omitted from the search results is audio, videoand/or A-V content. The reason for this is that the search enginesavailable to users today search the web pages of the internet bycomparing a text-based search string entered in query box 107 (FIG. 1 a)to the text that appears in different formats on the web pages. Thetypical formats that are searchable include extensible markup language(“XML”), hypertext markup language (“HTML”), portable document format(“PDF”), word documents (“DOC” and “DOCX”) and a number of others thatare recognized and regularly used by web programmers.

A problem that exists is that the standard search engines are notequipped to search anything other than text-based strings. In the pastfew years, as user bandwidth has increased, there has been an explosionof audio, video and A-V content made available to internet users on webpages. Anyone with a media player accessible through their computer caneasily access audio content such as songs, speeches, audio books,movies, or any other sound-based content as well as any audio that isincluded with video content, either in recorded or live form. Some ofthe most popular web pages on the internet such as the website YouTube(owned by Google, Inc.) are dedicated entirely to A-V content and allowusers to access A-V content uploaded by other users.

FIG. 2 a is an internet screen shot of a YouTube web page 201. To searchthe content media files on YouTube to find those of most interest to auser, the user types a text-based string into a query box 203. Thisprocess is very similar to the process described for a standard searchusing the Google search page in FIGS. 1 a-1 d. FIG. 2 b shows YouTubeweb page 201 with the search string “An Epiphany hit me. You gotta makesomething good” entered in query box 203. To find the most relevant A-Vcontent, YouTube employs a search algorithm that is based on atext-based description of the media file input by the person whouploaded it to the YouTube web site. The text-based description of thevideo input for user-uploaded video is typically limited to auser-designated title and may also include a short description of thecontent.

FIG. 2 c shows an example of results 205 a-d returned to the user basedon the query entered. There is no capability to search the actual audiotrack of the recordings available on YouTube to find and display thespecific words embedded in the recording itself. But the YouTube searchengine will deliver results that it determines are closest to the inputquery based on a text based title or description associated with therecordings. Once the results are returned, as with the Google search ofFIG. 1 a through FIG. 1 d, YouTube permits a user to click on any of thehits 205 a-d to be linked to the page corresponding to the selected hit.As an example, if a user clicks on hit 205 a for Angela Bofill “GottaMake it Up To You,” the user is taken to web page 207 (FIG. 2 d),allowing the user to access the video file identified in search result205 a. Once a user accesses web page 207, the user can play the selectedrecording from its starting point using a media player (such as a flashbased media player for example produced by Adobe Systems, Incorporatedof San Jose, Calif.) loaded on their computer (or residing on anaccessible network like the internet) represented on web page 207 asdisplay window 211, and having controls 209.

Controls 209 include a play button 209 a that allows the user to playthe video file by clicking on it. Once the video is playing, play button209 a becomes a pause button (not shown), allowing the user to pause thevideo at any time while it is playing. In pause mode, the pause buttontoggles back to being play button 209 a. A progress bar 209 b shows therelative elapsed portion of the video that has been played and isrepresented by a dot 209 b 1 that advances along progress bar 209 buntil the video is complete, at which time dot 209 b 1 will reach thefar right side of progress bar 209 b. Also within progress bar 209 is atimer 209 c showing both the elapsed time from the beginning of thevideo and the time remaining until the video is complete. Other buttonsto control the video include a volume control 209 d, a high-qualityswitch 209 e, a full screen switch 209 f that allows a user to watch thevideo on the full screen rather than inside the smaller framed displaywindow 211 within YouTube web page 207, and a closedcaptioning/annotations button 209 g for activating and de-activatingthose functions.

The video itself is shown in display window 211, and advertising 213 istypically shown on the right side of web page 207. The flash-based videoplayer of FIG. 2 d is one example of such a player. There are manyflash-based video players available and other such players may have adifferent appearance while offering similar functionality to play videoin different formats with or without a corresponding audio track.

It should be noted that the search tools and other applicationsdescribed in this disclosure for use on a desktop computer also apply toany other device, such as a laptop computer, a netbook, a mobilesmartphone, online televisions systems or any other device havingnetwork access and a web-browser with video capabilities available tothe user.

The invention recognizes that a user searching the internet is just aslikely to be seeking audio, video or A-V content as they are to besearching for standard text-based content, given that audio content,video content and A-V content are an alternative form of providinginformation to text in XML, HTML or other formats typically used todisplay information on a web page.

The advanced search capabilities of the present invention provide foruser search on the internet or across any other network, cloud computingenvironment, or electronic storage medium where audio-only, non-verbalvideo (e.g. sign language signaling in a video or the display of visualsigns or text in a video) and/or A-V media content resides and can bereturned along with standard text results. The invention may also beused with American Sign Language (“ASL”) or other non-spoken signals.While the invention uses transcription of the words spoken in an audiotrack in most instances, the invention can be adapted to search for textrepresenting any number of other visual or audio indicators embeddedwith the searchable content in a text-based form, including but notlimited to: (a) non-spoken language, in the case of ASL or other formsof sign language; (b) foreign language subtitles, which represent spokenlanguage, but not in the same language as the audio track; (c) soundeffects, which appear in parentheses or brackets in transcription andcaptioning for persons who are deaf or hard of hearing; (d) videodescription for persons who are blind, which is a secondary audio trackin which a “narrator” describes what is happening visually on the video;(e) description of visual events or signage present on the video, suchas the identity of a person speaking (the name of a famous personspeaking could be added by facial recognition software, for example, andthen become a searchable term), the image of a famous landmark,description of an event such as a flood or a bomb exploding, ordescription of an object appearing in the video.

The invention also works with supplemental text added to the transcriptfor ease of navigation of the video, such as “chapter markers,” toindicate a new topic or chapter in the video, “agenda items,” and/or“glossary terms.” Supplemental text such as glossaries or agendaspresented at meetings, lectures, business presentations or in othersettings may be provided using a hyper-link and may be searched byextension of a particular document. For example, where a user searchesfor a text sequence that appears in a title of the agenda or glossary,such as “introduction” or “follow-up,” they may be taken to the exactplace in the video or audio file where that text appears.

Glossary terms can be generated automatically (or manually) when a videois posted to a web page, for example, by software that culls the wordsappearing in the transcript and gives them a “rating.” Those terms areoften suggested as meta tags, and those tags can be searchable by addingtime codes to indicate each occurrence of the word in the video. As afurther example, if a video mentions the term “dysplasia” twenty timesand the term “encephalitis” fifteen times, those words are obviouslyimportant “glossary terms” that can be searchable. This could tie into aparticular page or paragraph about the search string and an indicatorsuch as an increase in font size or highlighting can be used toemphasize the importance of those terms.

Another way to use glossary terms is to cross reference synonymous termsto enhance the search and provide more valuable search results. Forexample, the search could access unique vocabularies or dictionariesthat may be used in a particular area of technology, or in a specificindustry. As an example, searching on the word “smack” could also returnresults for “heroin” and vice versa. An example of such a dictionary isthe urban dictionary which is found at:

http://www.urbandictionary.com/define.php?term=smack.

FIG. 3 a shows an internet screen shot of a search tool that is the sameas that shown in FIG. 1 a with a search string “An epiphany hit me. Yougotta make something good” entered in query box 303. FIG. 3 b shows thesearch results 305 including not just text-based search results but alsoany audio-only, video-only and combined A-V content that match thesearch string. When an audio-only, video-only or A-V result 305 isselected by the user from the list, the search string is passedautomatically from search box 303 in the search tool to a search box 313in media player 309 of the invention shown in FIG. 3 c allowing mediaplayer 309 to be queued for play of audio-only, video-only and/or A-Vcontent at the first occurrence in the content of the original searchstring. The audio-only, video-only or A-V content is also synchronizedto the time-coded corresponding text in a transcription display window321.

FIG. 3 d shows a “help screen” for a sample media player 311 of the typedescribed with respect to FIG. 3 c with descriptions for the differentparts of media player 311. Sample media player 311 is in the form of asoftware application that allows a user to accurately search a word orphrase using a standard search tool 301 such as Google or Bing. Once theresults for a search string are selected, the media content isauto-populated to be viewed in various ways: First, the search string intext form is shown in search box 313 and indicated by highlighting,italicizing, bolding or in any other manner along with the rest of thetext-based transcript in transcription display window 321; second, thesearch string in text form along with the rest of the text-basedtranscript is shown in captioning bar 315; third, a time line of themedia content is shown with one or more search string index markers 317indicating the points along the time line the search string occurs inthe content. The corresponding video content is shown in display window319 as the audio content is played and the text is shown intranscription display window 321.

The present invention allows a user to go beyond standard text-basedsearch and search audio content either alone or in sync with videocontent to find the specific occurrences of words within the audio,video or audio-video content. This is accomplished by providing atranscription of the audio content, embedding it with the searchablecontent, and making it available to be searched using a standardtext-based search field. When a user types a query into search box 313of media player 309, the matches are displayed in a number of ways. Thetext matches appear highlighted in transcription display window 321, andthe corresponding audio, video or audio-video is displayed synced tothat search string text result. The user plays the corresponding audioor video from that point in the file by pressing the ENTER key on thecomputer keyboard (see FIG. 6) or using the mouse to click on the PLAYbutton or anywhere on media window 309 or through an alternate inputsuch as a voice command. The user then has the ability to jump to thenext occurrence of that same search string within the content. This canbe accomplished in any number of ways. As an example, the user may pressthe RIGHT ARROW button or ENTER key on the keyboard, or provide a voicecommand. The corresponding audio, video or A-V is synchronized, so italso advances instantly. In addition, if the user reads ahead or back intranscript display window 321 and would like to listen to anotherportion of the transcript, the user simply clicks on that text in thetranscription display window 321, and the content file displayed inmedia window 309 automatically plays from that location.

Data mining of multimedia content is achieved by the invention, whichgenerates accurate search engine optimization (“SEO”) and social mediaoptimization (“SMO”) as a benefit to the web site content providers andowners making the content available on their web page(s). Examples ofsocial media sites are Twitter, Facebook, LinkedIn, MySpace, etc. Thesteps for performing SEO and SMO are shown in the Flowchart of PlayerProcess referenced in and further explained in FIG. 4 and FIG. 5.

It should be noted that there are numerous methods of transcription(including realtime transcription, realtime captioning and automaticspeech-to-text transcription) and numerous ways of generating time codesthat can be used to synchronize text and audio or video (such asrealtime transcription, realtime captioning, extraction of closedcaptions, and hand synchronization) that would be recognized by those ofordinary skill in the art in the field of transcription. The inventionrecognizes and accommodates these various methods.

It should be noted that there are numerous formats of audio and videofiles produced and in widespread use today. Conversion from one formatto another is common and the invention is compatible with any type offile. This disclosure is not intended to be limited to use with anyparticular file format.

FIG. 4 illustrates a representative process for displaying media contentin accordance with the present invention. Audio or video content isprovided at step 401. To maximize the efficiency and accuracy of asearch, it is preferable for the audio content to be accuratelytranscribed in step 403. If necessary, the audio or video is convertedto a video format compatible with the specific synchronization softwarebeing used (such as TimeCoder Pro by InData Corp. of Gilbert, Ariz.) atstep 405. Using the accurate transcript and the video file, thesynchronization software is used to produce a time-coded, (andcaptioning-friendly) text file at step 407. The text file is thenchecked for accuracy and synchronization at step 409 against the video.Following the correction of errors, if any, the file is then exported toan .XML file at step 411. The .XML file contains the text and the timecodes which correspond to elapsed time in the video. The video is thenconverted to the final video format required by the media player (inthis case, a standard .FLV web-based flash file format) also at step411. The video is then uploaded to the video server at step 413. The.XML file (as well as any supporting files that customize the look ofthe player, that display the transcript or the client logo, etc) areuploaded to any web site at step 413, along with the index.php andconfig.php files. The proprietary index.php and config.php files aremodified to point to the correct directories and corresponding files atstep 415. An example of the modification of the config.php file is asfollows:

  $config[′videos′][4][′video_title′] = ″David Gallo- UnderwaterAstonishments″; $config[′videos′][4][′video_url′] =″rtmp:/vod/DavidGallo2007.flv″; $config[′videos′][4][′xml_file′] = ″http://72.167.41.166/newclient/ted/davidgallo/DG042609.xml″;$config[′videos′][4][′pdf_download′] = ″http://72.167.41.166/newclient/ted/davidgallo/Underwater.pdf″;$config[′videos′][4][′client_logo′] = ″http://72.167.41.166/newclient/ted/davidgallo/RTIlogo.jpg″;

The index.php and config.php files cross reference one another anddetermine the design layout of the web site, as discussed below. In thisparticular process, the final steps are to clear the cache and test theplayer on different web browsers, submit to search engines and/or makesure the web site containing the video content gets indexed frequentlyat step 417. (Further illustrated and explained in FIG. 5, below.)

The flowchart of the player scripting process is shown in FIG. 5. Oncethe output of the .XML code is generated at step 501, the index.php andconfig.php scripting process are designed to receive updated time codeinformation reporting to the exact position reference point in thecontent file at step 503. This highlights, bolds, italicizes or in someother way indicates to the user the searched text string in thecorresponding text transcription display window 321 adjacent to theplayer in Javascript at step 505. This process is repeated as the playeris advanced line by line in the .XML code at step 507. The scriptadditionally duplicates the .XML data via a static HTML code that isnecessary for indexing and web crawling by popular search engines (e.g.Google, Bing and others) at step 509. While playing the content, theHTML code is modified to highlight the passage of the relevanttranscript phrase at step 511. Google or other Web search engines are bynature designed to crawl and index web sites that contain this staticHTML code at step 513. Once the search engine indexes the page, theresults are displayed for SEO (Search Engine Optimization) purposes atstep 513. The results are designed to bring the user to the playerwithin one click of the mouse, efficiently and seamlessly with thesearch query phrase auto-populated and ready to play at that referencepoint at step 515 and 517 respectively.

The .XML, JavaScript and flash .FLV files are uniquely used togetherduring playback in the following manner: As the content plays, atime-coded update is sent to the external script, reporting its exactposition in a progress bar that is the same or having similar functionto progress bar 317 shown in FIG. 3 d. The script receives thistime-coded information and highlights or displays the transcript textcontent in the .XML file that corresponds with the synchronized timecode in the content. (Note: The .XML file contains both the transcripttext and the time-coded information). A sample of the .XML file is asfollows:

  <TimeMs>18780</TimeMs> <PageNo>1</PageNo>  <LineNo>1</LineNo>   <QA>-</QA>  <Text>We are going to go on a dive to the deepsea,</Text>  </Line>   <Line ID=″1″> <Stream>0</Stream><TimeMs>22340</TimeMs> <PageNo>1</PageNo> <LineNo>2</LineNo> <QA>-</QA> <Text>and anyone that's had that lovely opportunity</Text> </Line>  <Line ID=″2″> <Stream>0</Stream>

Upon the content playing, the transcript display script will receiveconstant updates and will show each .XML content value that exactlycorresponds to the content as it is playing. Before the content isplayed, the complete transcript is additionally listed in static HTML tofacilitate search engine crawling/indexing for SEO purposes. Uponplaying the content or searching the transcript, this HTML code ismodified to highlight or otherwise indicate the relevant phrase to theuser. This static HTML is modified with industry standard coding inJavaScript. Upon a user clicking on a selected link in a list of searchresults 305, the player page will load, pull the key phrases from thesearch query box 303 using techniques understood by and commonly knownto those of ordinary skill in the art, and automatically perform aninternal search on the transcript for this same search string, and thenlist the search results at the top of the page (each are linked to theappropriate starting point within the media player.) The completetranscript will still be shown in the transcription display window 321below this box with the search string populated and indicated to theuser. Hitting the ENTER or TAB key or clicking on one of the searchstrings will take the user to the exact point in the content that isfound by the search string. Once the content is opened in the mediaplayer, the user can easily search content using the features describedwith respect to FIGS. 3 a-3 d above and further explained with respectto the help page of FIG. 3 d.

The $_SERVER[‘HTTP_REFERRER’] variable is used by developers for manydifferent reasons and applications. In the current invention, it is usedto search for audio, video and audio-video content to be easily browsedby providing the content as generic, textual search data that can beconsumed by any third-party search utility that operates by searchingtext. (e.g. Google, Bing, etc.). Or, as a means to detect what the usersearched for in the third-party search tool, and to automatically queuethe resulting content to best match that information.

One example of how this is accomplished is as follows:

-   -   a) A content file with embedded timeline information is provided        in .FLV. A textual transcript of the video/audio content is        provided with time-markers embedded in the transcript that        indicates the corresponding part of the content file in XML.    -   b) The transcript and content file are programmed such that the        text is linked to the specific point in the timeline that        corresponds with that transcript. An example of this is in FIG.        3 c showing the XML file contents and in the XML sample shown        above. The time in milliseconds is shown in the first line of        the XML file above as 18780 or 18.780 seconds (also see Appendix        A), and that time is from the start point of the content file.        This additionally can be referenced as page and line numbers as        described with respect to FIG. 5 above.    -   c) The complete transcript is displayed as plain HTML text so as        to be picked up by search engines and other third-party search        indexing tools.    -   d) Once the transcript has been indexed by the third-party        search tool (e.g. Google, Bing, etc.), using HTML, which search        engines are set to crawl automatically and index, it is included        in search results such as search results 305 in FIG. 3 b.    -   e) By default, when the user is on the results page (FIG. 3 b),        the URL of the results page reflects the search they just        performed, as in the above example:

http://www.google.com/search?hl=en&q=Dr.+Edith+Widder%2C+she+is+now+at+the+Ocean+Research+%26+Conservation+Association%2C+was+able+to+come+up+with+a+camera+that+could+capture+some+of+these+incredible+animals&cad=h

-   -   f) When a user clicks on the results link, they are taken into        the media player.    -   g) The system reads a server environment variable for that        request that tells the system the URL from where the user came.        (Ex: In a PHP file, this value is retrieved with the Javascript        command $ SERVER[‘HTTP REFERER’] variable contains this data).        In the case of the above search image, this value would be as        follows (the URL of the search string, the “referring” page):

http://www.google.com/search?hl=en&q=Dr.+Edith+Widder%2C+she+is+now+at+the+Ocean+Research+%26+Conservation+Association%2C+was+able+to+come+up+with+a+camera+that+could+capture+some+of+these+incredible+animals&cad=h

-   -   h) This value is parsed for the search query that the user typed        into the search engine. In this example, it was:        -   “Dr. Edith Widder, she is now at the Ocean Research &            Conservation Association, was able to come up with a camera            that could capture some of these incredible animals”    -   i) A secondary query is then automatically performed on the        transcript for the presence of this search string. Before the        system loads to the user, this search on the transcript is        performed, and the user is presented with the content with a        search already performed on the transcript. The parts of the        transcript that were found during this secondary search are then        provided prominently by, for example, auto-populating in the        search window 313 using highlighting, underlining, bolding,        italics and/or close captioning 315 over all other results. They        allow for the user to click on them and be taken to the precise        location in the video/audio where this searched-for string        appears as displayed in transcription display window 321.

Appendix A attached hereto and incorporated by reference herein shows atypical page source html code listing from a sample synchronized videotranscript.

FIG. 6 is a simple block diagram of a system, including a networked homecomputer 601, on which the invention may be implemented. Home computer601 is shown having a screen 603, a keyboard 605, a mouse 607 and acentral processing unit (“CPU”) 609. Home computer 601 is connected to anetwork 611 through a USB port (not shown), a wireless connection usinga wireless router 613 or any of a number of other connector ports thatare typically used on computers to connect them to outside networkseither directly or through a service provider using cable, satellite,phone or other service lines 615 coming into the user location. Theconnection to network 611 permits home computer 601 to have access tothe internet or another network where a user can conduct searches asdescribed throughout this disclosure. It will be understood that othertypes of devices with search capabilities can be substituted for homecomputer 601 as identified herein.

Home computer 601 is capable of executing software applicationsincluding search and media players as described herein. Suchapplications are executed on home computer 601 and may reside locally onhome computer 601 or at a remote location on network 611.

It is to be understood that the above descriptions and drawings are onlyfor illustrating representative variations of the present invention andare not intended to limit the scope thereof. Any variation andderivation from the above description and drawings are included in thescope of the present invention. For example, while a linear progress bar209 b has been shown in the figures, there are many different ways torepresent the progress of the audio-video content as it is played. Acircular depiction could be shown instead or any other visual techniquefor letting a user know the current position within the audio or A-Vtrack relative to beginning and/or end point(s).

What is claimed is:
 1. A method of conducting a search on an electronicdevice for finding and displaying results including content that istext, audio, video and/or audio-video, comprising: (a) inputting asearch string of text on the electronic device; (b) performing a searchusing the search string on content accessible to the electronic device;(c) finding search results wherein each of the search results includesthe search string in content and where the content is not text, atranscript is embedded with the content in text form; (d) providing alist of the search results to the user on the electronic device; (e)inputting a command on the electronic device to select a result from thelist that has content that is not text; (f) opening a media player onthe electronic device that displays a textual transcript of the contentwith the search string indicated to a user for the selected result. 2.The method of claim 1 further comprising showing a marker for at leastone location of the search text on an indicator at a point in the audiocontent corresponding to an occurrence of the search string.
 3. Themethod of claim 1 further comprising indicating an occurrence of thesearch string at each location where the search string appears in thetranscript.
 4. The method of claim 1 wherein the audio is queued tobegin play at a first occurrence of the search string.
 5. The method ofclaim 3 wherein the search string is indicated to a user by one or moreof the following: a) highlighting the text; b) italicizing the text; c)underlining the text; d) coloring the text; e) changing the font or sizeof the text; and/or f) bolding the text.
 6. The method of claim 1wherein upon opening a search result of content, the media player isauto-populated with the textual transcript synchronized to the contentwith the textual transcript queued to the start of an occurrence of thesearch string.
 7. The method of claim 1 wherein the content includes arecord of words that may be specified in a textual transcript and is atype from the group comprising: (i) audio content including words thatare presented through speech, including but not limited to words thatare spoken or sung; (ii) video content including sign language or othervisual cues that have defined meaning, including but not limited toAmerican Sign Language; (iii) supplemental text; or (iv) visual imagesof a recognizable person, place or thing that may be described orrepresented by text.
 8. The method of claim 1 wherein the electronicdevice is connected to a network, and further wherein the search isperformed on content on the network.
 9. A device for searching contentto find and display results comprising: (a) a central processing unit(“CPU”); (b) a display connected to the CPU for displaying information;(c) at least one user interface connected to the CPU for inputtinginstructions to the CPU; (d) a source of content including contentcomprising (i) text-based content, (ii) audio-based content, (iii)video-based content, and/or (iv) audio-video based content to which thedevice is connected and through which the content is accessible; (e) asearch application operable on the device by a user through the at leastone user interface that is executed by the CPU to search a string oftext embedded in the content and to present search results of content onthe display, wherein the content in each of the search results includeseither: (i) text based content with the search string; or (ii)audio-based content, video-based content or audio-video based contentwith the search string; and (f) a media player application operable onthe device by a user through the user interface that is executed by theCPU to play one of: (i) audio content, (ii) video content, or (iii)audio-video content, upon a particular search result having contentbeing selected by the user from the list of search results, the mediaplayer being opened and queued to begin play.
 10. The system of claim 9wherein the search string is indexed to at least one location on aprogress indicator with a marker at a point in the audio contentcorresponding to an occurrence of the search text string.
 11. The systemof claim 9 wherein a textual transcript of the audio content isdisplayed with the search string indicated to a user at each occurrencein the transcript.
 12. The system of claim 9 wherein the audio contentis queued to begin play at the first occurrence of the search string.13. The system of claim 11 wherein the search string is indicated to auser on the display by one or more of the following: a) highlighting thetext; b) italicizing the text; c) underlining the text; d) coloring thetext; e) changing the font or size of the text; and/or f) bolding thetext.
 14. The system of claim 11 wherein upon opening a search result ofcontent, the media player is auto-populated with the textual transcriptsynchronized to the content with the textual transcript queued to astart of an occurrence of the search string.
 15. The system of claim 11wherein the content includes a record of words that may be specified ina textual transcript and is a type from the group comprising: (i) audiocontent including words that are presented through speech, including butnot limited to words that are spoken or sung; (ii) video contentincluding sign language or other visual cues that have defined meaning,including but to limited to American Sign Language; (iii) supplementaltext; or (iv) visual images of a recognizable person, place or thingthat may be described or represented by text.
 16. The device of claim 9wherein the device is connected to a network, and further wherein thesearch is performed on content on the network.